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ABSTRACT 



This paper describes Innovation Configuration (IC) Mapping 
as an approach for assessing the extent of implementation of a program. An IC 
Map is similar metaphorically to a road map in that it summarizes different 
ways of getting from one point to another. Each map consists of basic units, 
or components, that can be made operational in a number of ways (variations) . 
Components can be combined into clusters. The focus in an IC map is on 
identifying and describing what one would observe in a setting in which the 
innovation is in use. IC maps are useful in many situations, including 
self -ref lection, peer observation, other observations, staff development 
diagnostics, document implementation, and personnel evaluation. An IC map 
does not tell a researcher how to collect the data. It is only a recording 
device. (SLD) 
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Well, I don’t think we will need to look at Innovation Configurations. There 
won’t be any adaptations. They are in their first year of use. (Phone call 
discussion with school district staff developer.) 



In recent years there has been greater recognition of the importance of having systematic 
knowledge about the extent of implementation before drawing inferences about effects. 

In the past, findings from summative studies frequently were reported that included little 
or no data about the way that “users” in the treatment group were performing. In 
addition, there frequently seemed to have been an implicit assumption that no members of 
the so-called “control/comparison” group could be performing in ways that closely 
resembled behaviors of those in the treatment group. Conversely, most seemed to assume 
that individuals assigned to a treatment group would be users. In many studies, there 
seemed to have been little effort to document directly what those in the treatment and 
comparison groups actually were doing, or not doing. When use was considered, the self 
report of group members that they were, or were not, using the program/process was often 
considered sufficient. The bulk of the report would quickly move to analyses of findings 
with a systematic search for differences in student outcomes. 

Today, there is an increased expectation for researchers and evaluators to document in 
some way that there was use of the new program/process before reporting on student 
outcomes. However, expectations still seem to be modest in terms of the amount of 
information about implementation that is to be reported. Perhaps guidelines and 
procedures for documenting use/non use in experimental and evaluation studies are 
needed? 

In this paper a particular approach is offered. Innovation Configuration Mapping, for 
assessing the extent of implementation. The approach has emerged from research on 
change in schools and professional development programs where teachers have been 
asked to use new curriculum approaches and new teaching strategies. The construct of 
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Innovation Configurations (IC) and its related measurement methodology has been used 
also in business and industry. Therefore, a variety of examples of applications of IC 
Maps in large and small school districts, and with small and large reforms , will be used 
to illustrate the methodology, data analysis procedures and the types of findings that are 
derived. An additional theme in this paper is to identify quality standards for acceptable 
IC Maps. 

In the next section of this paper a sampling of literature related to implementation is 
summarized. In the succeeding section the construct of Innovation Configurations is 
described in more detail. The methodology for developing IC Maps is presented next. 
Then data from recent studies and examples of IC Maps are presented to illustrate criteria 
for quality IC Maps and the types of data analyses that are typically applied. The paper 
concludes with a summary and discussion of selected implications. 

Adaptation, Reinvention and Configurations 

The idea that an innovation might be altered during implementation would seem to be 
common sense. However, discovery and addressing this phenomenon has been slow to 
crystallize with theorists, researchers and evaluators. In fact, it is difficult to identify any 
references to this possibility in the literature. One of the earliest, if not the first, clear 
data-based analysis of an educational innovation being used differently across teachers 
and classrooms was reported by Gallagher in 1967. He observed six classrooms where 
four high school science teachers were supposed to be using the same BSCS curriculum. 
Gallagher’s observed that each teacher’s practices were dramatically different from one to 
the other and that all varied in some ways from what the curriculum developers had 
envisioned. 

From an operational standpoint, this (sic) data would suggest that there really is 
no such thing as a BSCS curriculum presentation in the schools. Rather there is 
the URIAH interpretation of the BSCS curriculum, and the VIRGIL interpretation 
of that curriculum, and so forth. The substantial differences found in topics in 
terms of goals and levels of abstraction suggest that the teachers have different 
approaches in terms of insructional strategy that result in different ideas and 
concepts being presented to the students, page 1 7 

Rice and Rogers (1980) addressed this topic from a different perspective when they 
proposed the construct, reinvention. They took the frame of reference of the innovation 
to hypothesize that over time as an innovation is passed from adopter to adopter it could 
be changed or reinvented. Berman and McLaughlin (1978) in the RAND Change Agent 
study reported that the various innovations that they were studying were “adapted ’ at the 
host site. They then hypothesized a form of the change process, called mutual adaptation 
would be best. In other words, change would be most successful when the people 
changed some and the innovation was changed some to better match the local situation. 
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The writing of Rice and Rogers, and Berman and McLaughlin, led to the Fidelity 
Question: To what extent can, or should, the developers of an innovation expect users to 
implement the innovation exactly as prescribed by its creator(s)? A corollary question is. 
To what extent can an innovation be changed/adapted and still be considered the same 
thing? Which leads to the construct of Innovation Configurations. 

In 1977 Hall and Loucks proposed the construct of Innovation Configurations. This 
construct emerged from their experiences, and those of their colleagues, in one of the 
early studies of the Levels of Use diagnostic dimension of the Concerns Based Adoption 
Model (Hall & Hord, 1987; Hall & Hord, 2001). In this study, research team members 
interviewed teachers relative to their use of the innovation of team teaching, and college 
teacher education faculty relative to their use of instructional modules. The field work in 
both studies led the research team members to report back that although teachers and 
professors were saying they were “using” the innovation, their descriptions of the 
innovation and what they were doing under its name varied dramatically. In other words, 
there were different configurations of the innovation. This discovery was considered so 
important that assessing Innovation Configurations (IC) became the third diagnostic 
dimension in the Concerns Based Adoption Model (CBAM). Since that time a very 
useful mapping procedure has been developed to summarize information about each 
configuration, and increasingly IC Maps are being used to assess implementation in 
research and evaluation studies. IC Maps are being used also in assessing needs and 
designing training and staff development interventions. 

Innovation Configuration (IC) Maps 

An Innovation Configuration (IC) Map metaphorically is similar to a road map in that it 
summarizes different ways of getting from A to B. On a road map interstate highways, 
state highways and rural roads are identified using different colors and types of line 
segments. Each of the roadways represents one way of getting from A to B. Some will 
be quicker, others longer in distance, some more scenic and some may fit a certain mode 
of transportation better. The same is true with Innovation Configurations. Some ways of 
using an innovation will be quicker, others very elaborate, and some may better fit a 
particular setting or teacher disposition. An IC Map is a summary in words of the 
different ways the key components of an innovation can be made operational. Each 
configuration represens a differnt operational form. 

— Insert Figure 1 about here — 

An IC Map consists of a number of elements, each of which is represented in Figure 1. 
The basic unit in an IC Map is component. Each component identifies a particular 
operational aspect of the innovation. A component can be made operational in a number 
of ways. Each of these ways is called a variation. Components can be combined to form 
a cluster. Clusters are sets of components that describe a major theme or function of the 
innovation. 




3 



Figures 2, 3 and 4 present sample IC Map Components from different innovations. Each 
has been included to illustrate certain features that are indicative of quality in an IC Map. 

— Insert Figure 2 about here — 

The IC Map components in Figure 2 have been taken from the map of the Instructor’s 
role in Fast Track Ih a ten week course for entrepreneurs. When the Center for 
Entrepreneurial Leadership at the Kauffman Foundation, in Kansas City, decided to 
sponsor national dissemination of this course and use successful entrepreneurs as 
instructors, they realized that there would need to be training and quality control checks 
on the role of the instructor. An IC Map was developed and then used to identify the 
content for instructor training and to assess the extent to which the instructors followed 
the course plan. 

The IC Map components presented in Figure 2 were selected to illustrate the basic 
structural elements of IC Maps. Note that the number of variations for each component 
may vary. Component 2 has four variations, while Component 1 1 has 6 variations. 

There is no IC Mapping rule that sets the number of variations; this is decided for each 
component within a particular IC Map as it is being developed. We have observed, 
however, that four to six variations seems to be most common. 

— Insert Figure 3 about here — 

The components in Figure 3 were selected from the IC Map for Teaching and Learning 
Mathematics (Alquist and Hendrickson, 1999). These components illustrate more 
elaborate “word picture” descriptions for each variation. Note too that for this IC Map 
there are student components. Frequently, asking teachers to focus on components that 
address student behaviors can help teachers shape their practices in ways that will lead to 
students working in more ideal ways. 

— Insert Figure 4 about here — 

The IC Map component presented in Figure 4 is much more complex. This map was 
developed to describe classrooms where Essential Curriculum Principles are in place. 
Essentional Curiculum is a character education program. The philosophical foudation is 
a set of four principles that address how one behaves responsibly in a classroom and 
society. Since this is a heavily process and affective oriented innovation including more 
variations todescibe alternative operations for each component was important. 
Additionally, an antithetical variation was described and examplels of behaviors that 
could be reflective of a particular variation have been inclded as indicators. Each of 
these elements of the components are helpful in drawing easy to see word-pictures and to 
help in comaim different possible applications. 

— Insert Figure 5 about here — 
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Developing an IC Map 



A number of very important steps must be taken to develop a valid and effective IC Map. 
The key steps in this process are outlined graphically in Figure 5. An important caveat is 
that, in our experience, innovation developers are not effective in developing the IC Map 
for their product/process. There appear to be a number of reasons for this, which do not 
need to be discussed here. We have fount that IC Maps developed by others in 
collaboration with the creators of the innovation work best. 

The first phase of the IC Map building process begins with reviewing all printed and 
media materials related to the innovation. Then a series of interviews with the 
developer(s) is needed. Studing the materials in advance provides important background 
information and the context for listening and asking centered questions during the 
interview. The other element of data collection in this phase is to observe a range of 
settings where the innovation is in use. This can be done in the field or from videos. An 
advantage of observing in the field is the users can be asked directly about what they see 
use entailing. The objective from this phase is to identify the key components and 
clusters of components. Of course, along the way information about possible variations 
emerg as well. 

We have found that people use one of two ways to organize this early information about 
clusters, components and variations. The choice is a personal one. Some prefer a linear 
list or classification approach with topics and subtopics. This approach results in an 
inventory of potential components that is siimilar to the organization of a text book 
chapter with major sections, sub sections and sub sub sections. Other IC Mappers seem 
to prefer schema mapping or webbing. Each arm of the web comes to represent a cluster 
of components, or a single component. The twigs of each branch may become viariations. 
Either approach works well. The objective is to organize the information that is being 
collected in a way that will facilitate the identification of components, clusters of 
components and some of the variations. 

Once this Cluster Map has been drafted, work on mapping the first component can begin 
in earnest. We usually recommend that an IC Mapping team start with a core component, 
but one that is easy to see and describe. Do not start with one that is overly complex or 
subtle. Save these till the team has had some success in drafting IC Map components and 
variations. 

The word “seeing” was chosen deliberately above. The focus in developing an IC Map 
has to be on identifying and describing what one would observe in a setting where the 
innovation is in use. Two key questions to answer are ; 1) What would I see? And 2) 
What would people (teachers and students) be doing? The goal is to have component 
labels and each variation description as visual as possible. The objective is to develop 
clear word picture descriptions of the different ways each component could be used. 
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Once the IC Mapping team has a first draft of most components and variations it needs to 
be field tsted. We refer to this experience as the “dose of reality.” Rarely is there a case 
where a mapping team does not discover, with surprise and delight, aspects of practice 
that they had not been addressed in their early map draft. Following these first 
observations in live settings, or from viewing videos it is back to drafting and refining 
key components and their variations. This is an iterative process. Besides developing a 
useable 1C Map this process also develops a consensus view of what use of the 
innovation entails. 

It takes five (that’s right 5) days to develop a respectable complete first draft IC Map. 
Following this phase is the time of initial tryout by the IC Map team members and a few 
trusted colleagues. The feedback from these experiences leads to a second formal draft. 
Keep in mind that throughout there needs to be opportunities for the original developers 
to review the draft pieces and to confirm whether these are valid depictions of their 
innovation. 



— Insert Figure 6 about here — 

Checking IC Maps for Quality 

There are many technical details and subtle aspects that make for a useful and valid IC 
Map. A number of checks and tips for developing quality IC Maps are offered in Figure 
6 . 



— Insert Figure 7 about here — 



Using an IC Map 

There are many potential uses for IC Maps. A number of these are identified in Figure 7. 
Six of the most popular applications are: 

1) An IC Map is most useful for self reflection. A teacher, for example, can 
review what they have just done in a class period and assess their current practice by 
circling the variation of each component that describes what occurred. The teacher could 
then study some of the component variations that describe more advanced practice than 
s/he is currently doing. Moving from a “c” to a “b” variation might then become a 
personal goal. 

2) An IC Map can be used to guide peer observations. A colleague could 
observe using the whole IC Map, or focus on particular components only. Then the two 
could meet and discuss what was observed. As was described in the IC Mapping Tips and 
Guideline Figures, having the “d” and “e” variations describing alternative practices, 
rather than “not ‘a’” is critical if teachers are to place themselves at a point along the 
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continuum and to be able to see developmental steps, i.e., variations, from what they do 
now to what an ideal practice would be like. 

3) IC Maps can be an observation guide for principals and others who are 
expected to observe and coach those that are implementing an innovation. All too often 
principals, and others, are directed to “go observe your teachers.” They are not told what 
to look for or how to know when they see it. An IC Map then becomes a useful guide for 
focused observing. 

4) As a staff development diagnostic tool IC maps can be very influential in 
determining the content of training and professional development sessions. Critical 
components of the IC Map should become the content for training. If IC Map data were 
collected across a sample of teachers and it was determined that for one component most 
teachers were using “d” and “e” variations, than that component could be the topic for an 
in-service session. 

5) The important application of IC Maps that was identified in the introduction to 
this paper is to document implementation in research and evaluation studies. One of the 
authors has argued elsewhere (Hall, 1999) that it is imperative that implementation be 
assessed in every classroom before determining if there are meaningful differences in 
student outcomes. Assessing implementation needs to be done in comparison classrooms 
as well as in all “treatment” classrooms. 

6) A problematic application of IC Maps is in personnel evaluation. We 
steadfastly have argued against this application, although the siren of IC Maps being so 
relevant and useful for this purpose is hard to resist. It is not that IC Maps would not 
work well in evaluation, it is just that we have yet to work with an organization (in 
education or business) that was willing to invest the time to develop an IC Map to 
sufficient quality and to establish its validity so that its use in personnel evaluations 
would be defensible. Without doing the IC Map development work, the map will be of 
too low quality or lack creditability to be effective in personnel evaluation. Interestingly, 
in one project where personnel evaluation was the goal, as the IC Map started coming to 
life the executives in the organization saw that they too were going to be held 
accountable, not only for their skill development, but the continued professional growth 
of their subordinates. 

Analyzing IC Map data 

There are a number of strategies for analyzing IC Map data. They range from the simple 
to the elegant. At the simplest level the ratings of each teacher can be listed by 
component. A visual inspection to see where there are more “a’s” and “b’s” will lead to 
the identification of users that are most in line with what the developer(s) had described 
as ideal practice. These users could serve as deomonstration sites, while classrooms 
where there are more “d’s” and “e’s” could be provided additional technical assistance. 
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IC Map data also can be summarized by component thereby removing individual identity. 
In this approach it is use of the innovation that is the target. The frequency of rating of 
each variation is tallied and summarized by component and component cluster. This 
form of summary is useful for identifying components that most are using well and those 
where more attention may be needed. As was suggested in item 4 above, these data could 
be used to shape the content of a future staff developemnt session. 

For our research projects, Archie George has applied a hierarchical clustering technique. 
In this approach, a computer analysis is done to cluster together those individuals that 
have similar configurations. The clustering analysis begins by searching the data and 
identifying the two individuals who have the most identical component ratings. The 
analysis continues to group from there. In the end it is possible to examine clusters of 
subjects by IC Map ratings and to have them placed into high, medium, and low groups in 
terms of extent of implementation of the innovation. 

Once individuals, such as teachers, have been grouped according to the extent to which 
their use of the innovation matches the ideal, it is possible to compare them on other 
factors, such as student outcomes. In one recent major study, we were able to do this 
relative to implementation of a constructivist approach to teaching mathematics (George, 
Hall, Uchiyama, 2000). In taht study, classrooms with higher “fidelity” of use of a 
constructivist approach to teaching mathematics experienced higher student outcomes. 

Ideally, in a study of extent of implementation and student outcomes that used IC Map 
data it would be possible to identify which components of the innovation were associated 
with higher student outcomes. Then it would be possible to focus on those components 
for training and to seek their having the highest levels of implementation. Frequently, 
however, the quality and validity of the student outcome measures, the need for a very 
large sample size, and lack of a complete set of IC Map data limit the findings from any 
one study. 



Discussion and Implications 

The primary purpose of this paper was to introduce the construct of Innovation 
Configurations (IC) and the methodology for describing and assessing IC. As with any 
effort that brings together theory, research and practice, there are more points that need to 
be addressed than are appropriate for a paper of this type. For the reader who is interested 
in learning more, please contact the authors or one of their colleagues who are 
experienced with developing IC Maps. There also is a chapter in the book. Implementing 
Change: Patterns, Principles and Potholes, that examines IC in-depth. A few points that 
have major implications are identified and discussed briefly here. 

1) Treatment and comparison groups are likely to contain a wide range of configurations. 

Regardless of the amount of training and extent of quality control efforts, we have yet to 
see a situation where there were not individuals in the so-called treatment group that 
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were, at best, using very little of the components of the innovation. The data are clear, 
without direct assessment of the extent of implementation at the individual level, there is 
a great deal of risk in assuming that all are users. There are corollary risks in assuming 
that the comparison/control groups do not have individuals that are engaging in practices 
that are very similar to those identified as user variations in the Innovation Configuration 
Map. 

An alternative approach, which we have used with great success, is to not assign 
individuals to use/non use groups, a priori, based on whether or not they have received 
training and/or materials. Instead, assign individuals to groups post hoc based on the IC 
Map data cluster analysis. Place individuals into user groups based on the proportion of 
their IC Map ratings that are clustered around “a/b” variations, versus those with mainly 
“c/d” variations, versus those with mostly “e/f ’ variations. Then the study question 
becomes one of comparing high level implementations to medium and low levels. 

2) A burning issue related to IC Mapping is the issue of fidelity. 

Many argue against asking teachers to use a practice or program in a specified way. 
Interestingly, we regularly meet innovation developers who refuse to say that one way is 
better than another. They are apt to say, “It depends. That could be good, or not. It 
depends on what else is going on...” There seems to be a feeling that asking teachers to 
teach in a certain way, or to use certain materials is a professional violation. If this 
reasoning is carried to its extreme anything that is done would have to be included under 
the umbrella label of the innovation. 

Which leads to a dilemma, if there are no boundaries to what teachers can do, is there an 
innovation? Most innovation developers do have in mind practices that they believe will 
be best. For a number of reasons I would argue that the best form of honesty is to be up- 
front in telling teachers what those practices entail. A game of deception and discovery 
by trial is not treating teachers as professionals or honestly. It also causes resistance to 
change. 

A related problem is that in nearly all change efforts there will be multiple change 
leaders. Even though they may believe they are on the same page, the IC Mapping 
process consistently unveils that the mental images and implicit definitions each leader 
holds about ideal use can be different in critical ways. The unintended result is that each 
leader communicates to the prospective users a slightly different image of what ideal use 
entails. One of the benefits of the IC Mapping process is that the change leaders develop a 
consensus about what the ideal practice looks like. This not only results in the IC Map 
reflecting this, but once the consensus has been constructed, all of the leaders convey the 
shared image to their clients. 

One implication of this discussion point is that in-fact innovation developers do have in 
mind particular practices that they associate with the ideal. However, an IC Map does not 
have to include a fidelity perspective. The variations of each component could be 
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displayed in any order. Still, when there is a fidelity perspective there is an effective way 
to display this information on the IC Map by ranking the variations from “a” to “x.” 

3) An IC Map does not tell one how to collect the data, it is a recording device only. 

Since an IC Map presents word-picture descriptions of different ways that components of 
an innovation can be made operational there is an implication that the only way to collect 
data is by direct observation. Although observation certainly is preferable, there may be a 
number of components where an interview could be used to collect the data. It also is 
possible to have teachers rate themselves, although for some components there are likely 
to be questions of accuracy in the self-ratings. Again, data to be recorded on an IC Map 
may be collected in a number of ways besides observation. 

In conclusion, acknowledgment should be made that the authors did not define the term 
innovation early in this paper. Oue technical definition could be, “a process or product 
that represents change from current practice.” The innovation could represent something 
that is radical and new to all, or its name could be simply a place holder for talking about 
a current or traditional program. IC Maps could be developed for any or all practices, 
new or old. The major point here is that there is likely to be a range of applications and 
that everyone will not be doing the same thing. They may all say they are using “it.” 

They all have been through the same training program, and they may have “its” materials. 
But without direct assessment of practice for each individual there is little chance of 
knowing that all of the “its” are the same in practice, nor will each likely result in the 
same effects. 
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Figure 1. Elements of IC Maps 
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Note: This is a dynamic/interactive, concensus building process. 
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Figure 6 



IC Mapping Checks, Tips and Guidelines 



IC Mapping Quote : 

“Well, I don’t think we will need to look at IC. There won’t be any adaptations. They are 
in their first year of use.”(in phone call) California graduate student. 

Checkine Variations 

1) Watch out for overly simple “a lot” to “a little” variations 

2) Try to have more then one dimension in each component 

3) Watch out for components with too many (greater than four) dimensions 

4) Strive to have the wording of each variation as descriptive and observable as possible, 
i.e., word pictures 

5) The last variation to the right should not simply be “not” doing the “a” variation. 
Describe what is being done instead of “a” 

6) Be sure “e” and “f’ variations are not negative or pejorative. They should be 
alternatives to what has been described in the “a” and “b” variations. 

7) Do not type variations in boxes. This implies that each variation is finite and eliminates 
in between possibilities. 

8) “a” and “b” variations should describe the best, the ideal, but do not make them sound 
impossible to achieve 

9) Don’t forget to use checklists instead of “’a” to “e” variations where useful 



Checking Components 

1) Have components labels that are descriptive, not just a single word or topic, i.e., don’t 
just name a noun. 



O 
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2) Note which components will require observation (s) and which could be assessed via 
an interview. Mark those where an interview will be needed with a note or an asterisk. 

3) Determine and mark the components that are most critical. Use bold font for these. 

4) How unique is each Cluster and each Component? Mapping should be only for those 
clusters and components that are really worth it. 

5) Be sure to have rich students components along with the teacher/user components. 



Figure 7 

Uses and Purposes of IC Maos 

Articulation and description of what use can and should look like can lead to more 
implementation success, sooner, and (when desired) with higher fidelity. 

Develops consensus about what use looks like 

Word picture descriptions that can be used in self assessment 

Planning staff development and training 

Evaluating zva innovation’s effects 

Unstacking a bundle of innovations 

Assessing degree of implementation 

Focusing for coaching 

Helps communicate expectations of the dream 

Help visualize what can be done 

Clarifies alternatives so that each may be considered 

Provides diagnostic data for designing and targeting interventions 

Changes in intentions and expectations during implementation can be assemble 



AEKA 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 



TM031780 



The use of .'.Innovation Configuration Maps in assessing implementation: 
The bridge between development and student outcomes.! 


Author(s); Gene E. Hall, University of Nevada, Las Vegas & Archie A. 


George Univ. of Idaho 


Corporate Source: „ 




University of Nevada, Las Vegas 


Publication Date: 


College of Education 






April 2000 



monthly abstrw* Jounwl of the /ri^« community, documents announced in the 

and electronic media, and sold through the ERIC Document Rep^iS's^ndce (QORsf^ th '»P'«‘u«d paper copy, 

reproduction release is granted, one of the fblloiaing notioes IsSS^ ^ document, and. if 



of the page. 

The lample Mehar ahoMi batow «■ be 

idbiBdieelLeeiitdocumenti 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



Jb^. 









5 TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



The empto tUdcar eMMin below wHI be 
affixed to el Lewi 2A<ecumenli 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 

'N euctronic media 

FOR ERIC COLLECTION SUBSCRIBERS ONLY 
HAS BEEN GRANTED BY 



<b^. 









TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



2A 



LtvMI 

t 



m 



Ctw* hara tor UvM 1 raiaese, p«in^ 
ra^Mcflon and dhaamfeiNlon bi n4oral^ 

ERIC «rMmI madto (a^, alsdranie) antf paper 
copy. 



tree options and sign at the bottom 



The sample stickar ahoam twfow wfll be 

efrUj^to aU Level 26 *<«**«»nenli 

PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTED BY 












TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



2B 



Level 2A 

t 

□ 



Level 2B 

t 

□ 



Check hare tor Laval 2A ralaasa, pennlttiin 
wprediKaon and ifN aandna llu ii In fitee^ 
ale clr oiil t madia tor ERIC archNalco ll aclio n 
nbaoribare only 



Check hare for Laval 2B relaase. pemUttlng 
f<NmdiJdiQn and disaamina^ 






Sign 
here,-* 
please 

ERIC 





*^^vepemUssiori to mpmduM and disseminate trrisrkximent 
e<Ba by persons other then ERiC empioyees and its system 
^non-prx)mfBpfX)ductk)n tyUbfBriesmiothersefvioBQgencies 


oontm^nN^ BteepUanIsmadefb 

Psaas»intofmabcw/»ecteof6duc^^ in^julrfes. 




PrtnlidNsmaffusiUuiimtle: 

Gene E. Hall, Dean. L.- ' • , 


: 

College of Education 

453001 


TNM«fc-.(7Q2;)i895_3373 


''**(702)895-4068 


^^e*ffa*Hf^hevada • edu 


““ 8/04/00 



(over) 





Clearlnglioese on Assessmemtt amdl EvatoattidDim 



March 2000 

Dear AERA Presenter, 



University of Maryland 
1129 Shriver Laboratory 
College Park, MD 20742-5701 

Tel: (800) 464-3742 
(301)405-7449 
FAX: (301) 405-8134 
e ricae @ ericae. net 
http://ericae.net 



Congratulations on being a presenter at AERA. The ERIC Clearinghouse on Assessment and 
Evaluation would like you to contribute to ERIC by providing us with a written copy of your 
presentation. Submitting your paper to ERIC ensures a wider audience by making it available to 
members of the education community who could not attend your session or this year's conference. 

Abstracts of papers accepted by ERIC appear in Resources in Education (RIE) and are announced to over 
5,000 organizations. The inclusion of your work makes it readily available to other researchers, provides a 
permanent archive, and enhances the quality of RIE. Abstracts of your contribution will be accessible 
through the printed, electronic, and internet versions of RIE. The paper will be available full-text, on 
demand through the ERIC Document Reproduction Service and through the microfiche collections 
housed at libraries around the world. 



We are gathering all the papers from the AERA Conference. We will route your paper to the 
appropriate clearinghouse and you will be notified if your paper meets ERIC's criteria. Documents 
are reviewed for contribution to education, timeliness, relevance, methodology, effectiveness of 
presentation, and reproduction quality. You can track our processing of your paper at 

http://ericae.net. 

To disseminate your work through ERIC, you need to sign the reproduction release form on the 
back of this letter and include it with two copies of your paper. You can drop of the copies of 
your paper and reproduction release form at the ERIC booth (223) or mail to our attention at the 
address below. If you have not submitted your 1999 Conference paper please send today or 
drop it off at the booth with a Reproduction Release Form. Please feel free to copy the form 
for future or additional submissions. 



Mail to: AERA 2000/ERIC Acquisitions 

The University of Maryland 
1 129 Shriver Lab 
College Park, MD 20742 



Sincerely, 




Lawrence M. Rudner, Ph.D. 
Director, ERIC/AE 



ERIC/AE is a project of the Department of Measurement, Statistics and Evaluation 
at the College of Education, University of Maryland. 



